Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource Format Auto Detect #1369

Closed
wants to merge 7 commits into from
Closed

Conversation

JVickery-TBS
Copy link
Contributor

feat(schemas): use core guess_format for resource format schema;

  • Made resource format field read only.
  • Used core if_empty_guess_format validator for the field.
  • Modified choices into openness_scores for the Openness Score mappings on the format values.

I think we would like to have a custom validator, as the core one is if_empty_guess_format. I would like to keep this field read only as it seems like so many users do not select or type in the correct file formats.

So there are the scenarios now that if a user cleared an upload or change it to a link, then the if_empty_guess_format would not fire off again hence the if_empty. So doing a guess_format which would just always set the format correctly.

If the resource is a link and not upload however, what should we set the format to?

- Made resource format field read only.
- Used core `if_empty_guess_format` validator for the field.
- Modified choices into `openness_scores` for the Openness Score mappings on the format values.
@wardi
Copy link
Member

wardi commented Apr 18, 2023

Need business team support for this kind of a change, maybe you could demo it at dev chat?

@JVickery-TBS
Copy link
Contributor Author

Just putting down some stuff we talked about:

  • guess_format validator might not be a great idea, as this would prevent users from actually changing the value of it in the case of the validator guessing the incorrect format.
  • readonly on the field is also not a great idea for the same reason above.
  • core uses /api/2/util/resource/format_autocomplete for the select2 field input. Although this only autocompletes to existing format values in the database. I think we still would want to limit the format values to our old choices. But it would be best to put these into a select2 field input along with our own autocomplete to a REVISED list of allowed formats.

In all, we want it to guess the format automatically, but then we also want users to be able to override this by being able to use a select2 input field to autocomplete to a specific list of allowed formats.

The select2 field input would be a UX improvement and limit any possible misclicks in the current, long select list we have.

This means that we do not need the format field to be required as it will validate and guess a format if empty. On the other hand, is this clear to the user in our current UI (need to add any messaging to the format field description?)?

Note: the current openness score is already handled in this PR for any of the changes.

…idator for resource format;

- Added select2 scheming form snippet for specific display in our theme.
- Added override action method for `format_autocomplete`.
- Added custom validator to do format replacements based on `if_empty_guess_format` and `replaces`.
@JVickery-TBS
Copy link
Contributor Author

@wardi okay I worked on this concept a bit more and I think it is in a good place to show.

The format field is now a select2 with the normal endpoint to the format autocomplete. However, I override the callback to get the choice values from our schema and match those without case sensitivity.

And from my initial test, I was uploading a JPG image, and the if_empty_guess_format validator was setting it to JPEG which we do not have in our choices. So I used the existing replaces we have in our schema, and made a validator which matches against those. E.g. if_empty_guess_format would fire off and return JPEG, that would go through the new validator and match the replaces: [jpeg] and return JPG. We would probably have to keep adding to the replaces lists over time.

I have also removed the EXE format from the choices during this.

- Added labels and replaces checks for select2 callback.
- Added TODOs.
@JVickery-TBS
Copy link
Contributor Author

As discussed, we are going to plan to get some front-end stuff happening. Which includes:

  • localizing scheming options to JS
  • using JS callback for select2
  • not allowing select2 custom input
  • clearing format field when clearing upload or change of upload/url field
  • guessing format during upload/url field update
  • run guess format through scheming replacement matches
  • run scheming choices validation
  • simplifying front-end error message for format as it will have a select2 now

We will also have to confirm that the guess format works in the API. We should not change/simplify the API error message however, as that will NOT have a select2/type-ahead. We will also want to do something about the clear_upload in the back-end as well.

# Conflicts:
#	ckanext/canada/plugins.py
#	ckanext/canada/validators.py
# RESOLVED.
- Validator for guessing the resource format.
- Started frontend code for resource format.
- Changed autocomplete to basic selectable select2 element.
- Moved some code to reusable helper.
- Added API util endpoint.
- Test for auto resource format.
@JVickery-TBS JVickery-TBS deleted the feature/auto-resource-formats branch January 26, 2024 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants